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Researchers at the National Institute for Excellence in Teaching (NIET) and elsewhere have studied the 
effectiveness of TAP: The System for Teacher and Student Advancement in raising student achievement, 
improving the quality of instruction, and increasing the ability of high-need schools to recruit, retain, 
and support effective teachers. This document describes some of the most recent results that have 
emerged from the research on the TAP System to date. Data collection and analysis efforts are ongoing, 
and the findings described here will be updated periodically as information becomes available. 


NIET Mission 


Recognizing that an effective teacher is the most important school-based factor impacting student 
achievement, NIET is committed to ensuring a highly skilled, strongly motivated, and competitively 
compensated teacher for every classroom in America. NIET supports states, districts and schools in 
recruiting, developing, supporting, and retaining high-quality human capital in order to raise 
achievement levels for all students. NIET seeks to accomplish this commitment through two signature 
initiatives TAP™: The System for Teacher and Student Advancement and the NIET Best Practices Center. 


TAP: The System for Teacher and Student Advancement Description 


Introduced in 1999, the TAP System has become America’s leading comprehensive educator 
effectiveness model that offers powerful career advancement and leadership opportunities for 
educators, a fair and transparent evaluation process that is linked to job-embedded professional 
development, and performance-based compensation. Each of these core elements is discussed below. 


For more information, visit www.niet.org/tap-system. 


e Multiple career paths. In TAP schools, skilled teachers have the opportunity to serve as master and 
mentor teachers, receiving additional compensation for providing high levels of support to career 
teachers and increasing instructional effectiveness across the faculty. Master and mentor teachers 
form a leadership team, along with administrators, to deliver school-based professional support and 
conduct evaluations with a high level of expertise. 


e Ongoing applied professional growth. Led by master and mentor teachers, TAP teachers participate 
in weekly cluster group meetings where they examine student data, engage in collaborative 
planning, and learn instructional strategies that have been field-tested in their own schools. 
Teachers benefit from a national TAP database of instructional strategies and their colleagues’ 
experiences. Professional development continues in the classroom as master teachers model 
lessons, observe classroom instruction, and support teachers’ pedagogical improvement. 


e =Instructionally focused accountability. TAP teachers are observed in classroom instruction several 
times a year by multiple trained observers, including principals and master and mentor teachers, 
using rubrics for several dimensions of instructional effectiveness. Evaluators are trained and 
certified, and leadership teams monitor the reliability and consistency of evaluations in their 
schools. These classroom evaluations are complemented by analyzing student achievement growth, 
rounding out a multi-measure system of teacher evaluation. Evaluation results are used as formative 
feedback in one-on-one mentoring sessions and guide planning for cluster group meetings. 


e Performance-based compensation. TAP teachers have the opportunity to earn annual bonuses 


based on their observed skills, knowledge and responsibilities, their students’ average achievement 
growth, and school-wide achievement growth. Master and mentor teachers receive additional 
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compensation based on their added roles and responsibilities, and principals can earn additional 
compensation based on school-wide achievement growth and other measures of effectiveness. 


Best Practices Center Description 


The NIET Best Practices Center (BPC) provides innovative services, support, and solutions to schools, 
districts, and states to improve educator effectiveness. Based on more than a decade of experience in 
schools across the country, the BPC works with its partners to redesign educator evaluation, deliver 
effective professional development, implement performance-based compensation systems, and train 
teacher leaders in schools. The BPC offers a network of expert trainers and access to a range of 
innovative Web-based resources and tools to support individual initiatives and goals. For more 
information, visit www.niet.org/best-practices-center. 


TAP Influence in Schools 


Since its inception in 1999, the TAP System of comprehensive educator effectiveness has operated 
across multiple states in hundreds of schools. TAP has grown steadily in the number of schools 
participating, with over 90% identified as high-need. In addition to schools and districts implementing 
the full TAP System, a number of states, districts, and schools are using TAP System materials and 
practices—in particular, the online portal which houses hundreds of hours of effective teaching across 
subjects and grade levels, the TAP teacher observational rubric, and the certification and recertification 
process for educator evaluators. As of the 2014-15 school year, NIET initiatives are impacting over 
200,000 educators and more than 2.5 million students, approximately 5% of the American public school 
student population. 


TAP Impact in Schools and Districts 


A notable success of the TAP System is the expanding preponderance of performance data that comes 
from examining the impact of the system across multiple locations and conducted by different 
researchers using varied methodological frameworks (Algiers Charter School Association, 2011; Barnett, 
Rinthapol, & Alexander, 2015; Barnett, Rinthapol, & Hudgens, 2014; Buck & Coffelt, 2013; Daley & Kim, 
2010; Hudson, 2010; Schacter et al., 2002; Schacter, Thum, Reifsneider, & Schiff, 2004; Schacter & 
Thum, 2005; Solmon, White, Cohen, & Woo, 2007). The next section discusses several of these studies 
demonstrating the impact of the TAP System. Following this general review, three new reports and the 
results from the 2014 TAP Teacher and Administrator Attitude Survey are presented. 


Instructional Measurement That Leads to Improved Practice 


To improve the quality of classroom instruction, the quality of each teacher’s instruction must be 
assessed. Traditional school systems have not been successful at measuring and assessing classroom 
instruction. The New Teacher Project (TNTP) published a revealing report in 2009 showing that schools 
fail to evaluate their teachers in any meaningful way (Weisberg, Sexton, Mulhern, & Keeling, 2009). As 
TNTP reported, most teachers were rated at the very highest levels (replicated as Figure 1 below), 
despite the fact that most schools were not performing at these highest levels on achievement 
indicators. 


A 2014 report from the National Council on Teacher Quality (NCTQ) indicated that while improvements 
in educator evaluation have occurred since the 2009 report, still many of the same problems persist. 
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Given that differences in teacher effectiveness represent the single most important school-related 
factor affecting student learning, accurately measuring differences in teacher performance is critical to 
the improvement of teaching and learning. 


70% 
60% == 
50% = 
0, i 
Percent of ue 
Teachers 30% = 
20% +-——— -— 
10% =-—J +-——— -— 
0% 
Lowest Middle Highest 
Ratings Ratings Ratings 


Teacher Evaluation Rating 


Figure 1: Traditional teacher evaluation scores.1 


In contrast to traditional evaluation methods noted above in Figure 1, the TAP System has developed a 
comprehensive approach to teacher evaluation and incentives that depends on multiple measures of 
both teaching practice and teaching outcomes. This system provides differentiated feedback for teacher 
improvement, in contrast to the often found inflated ratings found in evaluation systems (see Figure 2). 


1 Teacher evaluations in five urban school districts, based on data taken from 
http://widgeteffect.org/downloads/The WidgetEffect.pdf. Scores on 3-point and 4-point scales have been 
interpolated to a 5-point scale using a cumulative probability density function based on reported data. 
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Note: Teacher Skills, Knowledge, and Responsibilities distribution of TAP evaluations using 1-5 scale in 
half-point increments. Figure based on 13,891 teachers and approximately 55,564 observations, 2013-14. 


Figure 2: Observational ratings of teachers in TAP schools. 


The above ratings are based only on the classroom evaluation component of the TAP System, before 
considering student learning growth measures. Teachers are observed several times a year by multiple 
trained and certified raters who consider 19 areas of effective instructional practice. These observers 
use a multi-dimensional, research-based set of standards and rubrics that are fair, transparent, and 
curriculum-independent. Results are provided immediately as feedback to the teacher in post- 
observation mentoring sessions. The scores from all observations of these 19 classroom indicators are 
combined with seven responsibility indicators at the end of the school year to create an overall Skills, 
Knowledge, and Responsibilities (SKR) score for each teacher. On a scale of 1.0 to 5.0, 1 represents 
unsatisfactory performance on a certain standard, 3 represents proficiency on that standard, and 5 
represents exemplary performance on that standard. Teachers earn scores in increments of 0.5. 


TAP Teachers Demonstrate Consistent Improvement across Time 


TAP results show a steady improvement in observed skills during the course of the school year. Figure 3 
shows recent improvement for teachers in TAP nationally. This shows the improvement in instructional 

quality scores over a two-year period. In the data shown, despite a slight dip over the summer, teachers 
demonstrated an overall path of improvement that continued over both years. This graph is based on a 

sample including all TAP schools during the years 2012-2014. 


We tracked a cohort of 5,469 teachers through observations grouped into six periods in fall, winter and 
spring of the 2012-2013 and 2013-2014 school years. The cohort was composed of teachers working in 
TAP schools both years, with observations in each of the observation periods. Teachers present in only 
one school year or who lacked an observation in one of the quarters were excluded from the sample. 
Including teachers present in only some of the periods would have complicated efforts to compare the 
level of instructional quality at different time points, since each average could reflect substantially 
different groups of teachers. 
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Figure 3: Teacher instructional improvement across time. 


Within the TAP System, improvements to teacher instructional practices translate into gains in student 
achievement. More importantly, these improvements increase over time. 


90% 
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Percentage 82% 
of Schools 80% 
Achieving a Year’s 
Worth 78% 
of Growth 


76% 


74% 


72% 


70% 
Control School TAP - New TAP - Continue 


Note: Figure created with raw school-level, composite math and reading score data provided by SAS® for 
control (n=3,870) and TAP schools (n=353). 


Figure 4: Student achievement growth by TAP System status 
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With student achievement gains and high performance scores on the teacher observation rubric, thus 
demonstrating increases in effectiveness, teachers are able to earn performance-based compensation. 
To represent this impact, Figure 4 shows how the salary of an effective teacher (defined as obtaining 
one year or more of student achievement growth and/or being rated as proficient on the TAP 
observation rubric, a top 10% effective teacher, and a TAP mentor and master teacher compares to all 
other teachers. 


As demonstrated by the left side of the graph, teachers in schools with no additional financial awards for 
improving the performance of students or improving their own practice as demonstrated by high scores 
ona teacher observation rubric would receive the same salary (reflected on Figure 5 as the average 
teacher’s salary nationally). However, on the right side of the graph, we see that in the TAP schools 
effective teachers can earn an average bonus of approximately $2,250 (4% of their annual salary), while 
teachers performing in the top 10% (those growing their students’ performance by significantly more 
than one year’s growth and demonstrating exemplary classroom practices) earn an average bonus of 
approximately $3,750 (7% of their annual salary). Further, in TAP schools, the multiple career paths 
allow for master and mentor teachers to earn approximately $10,000 (18% of their annual salary) 
through their performance bonus and salary augmentation for increased responsibilities. 


$68,000 
$64,000 
Non-TAP Schools TAP Schools 
$60,000 
Average $56,000 
Teacher _ 
- ™ i a Zz ae a 7 
$48,000 
Other Effective Top 10% Other Effective Top 10% Master & 
Teachers Teachers Teachers Teachers Teachers Teachers Mentor 
Teachers 


Figure 5: Teacher salary impacts based on demonstrated effectiveness. 
Results from Prior Selected Studies of TAP’s Impact 


e Mann, Leutscher, and Reardon (2013) examined the impact of the TAP System across fifteen 
schools in Louisiana. In order to determine impact, a one-to-one nearest-neighbor matching 
algorithm with replacements was created to find a comparison school for each TAP school. 
Based on the propensity scores computed using the selection model, the algorithm chooses the 
non-TAP school with the propensity score closest to the propensity score of the TAP school. 
There was no significant difference between the TAP schools and their matched comparison 


? Average teacher salary based on reported national average from the Bureau of Labor Statistics, 2012 
http://www.bls.gov/ooh/education-training-and-library/home.htm 
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schools in the pretreatment year, t(26) = 0.080, p > 0.05. However, results of the study indicate 
that in the four primary subjects assessed, there was a significant effect in favor of the TAP 
schools for ELA: F(1, 6421) = 6.334, p = 0.012; Mathematics: F(1, 6421) = 86.386, p = 0.000; 
Science: F(1, 7084) = 31.792, p = 0.000; and Social Studies: F(1, 7085) = 87.411, p = 0.000]. 
Further, the study examined the impact of the TAP System across time to find that the TAP 
schools significantly outperform comparison schools, F(1, 24) = 5.30, p = 0.031. The study also 
found that 92% of teachers reported that the TAP System made a positive difference on student 
achievement in their school and 91% reported that the AYP status was improved as a result of 
the TAP System. 


e In 2010, Hudson examined the effect of the TAP System on student achievement across 151 
schools in 11 states. Hudson used a statistical control matching method to ensure that the TAP 
schools and the comparison schools were equivalent prior to the intervention being 
implemented. Hudson also used a differences-in-differences approach to further account for 
any differences between the groups and to ensure that the evaluation was able to isolate the 
impact of the program. Results of the study indicate that students in TAP schools outperformed 
students in comparison schools by approximately 0.15 standard deviations in mathematics, and 
smaller effects but in favor of the TAP schools in reading. Hudson explains these findings in 
context to other education interventions by noting that “the estimated effect of TAP on 
mathematics achievement is more than twice as large [as class size reduction effects]” (p. 28). 


e In 2007, Solmon et al. analyzed the impacts of the TAP System in terms of value-added gain 
scores across 650 classrooms in six states, including 61 TAP schools and 285 control schools. 
Researchers analyzed the student achievement gains at two levels of comparison—teacher-to- 
teacher and school-to-school. To evaluate TAP teachers (and similarly in evaluating TAP schools), 
researchers calculated the effect of each teacher on student progress as assessed by the 
difference between the actual average scores of the teacher’s students and the expected 
average scores of those students (as derived from previous scores). Through this process, 
researchers created a statistical control group for the TAP teachers based on performance. 
Results of the study indicate that in every state more TAP teachers demonstrated statistically 
significant at or above average amount of student achievement growth than control group 
teachers. Further, TAP schools outperformed their controls in 57% of the categories in math and 
in 67% of the categories in reading. 


e Intheir 2002 study, Shacter et al., analyzed the growth in achievement of students (n=3,319) 
whose schools implemented the TAP System compared to the growth of achievement of 
students (n=7,055) from matched comparison schools. The schools were matched on 
achievement (percentile rank in reading, mathematics, and language), school size, percent of 
students eligible for free lunch, school configuration, and location. A statewide cluster analysis 
was conducted to match the schools. Beyond the matched comparisons, the results in 
achievement were based on a multi-level value-added model utilizing prior test scores as 
covariates. Results of the analysis revealed that TAP schools made significantly higher 
improvements in student achievement gains. Further, this study found that those schools that 
implemented the TAP System with higher fidelity more significantly outperformed comparison 
schools. 


e In their 2004 follow-up study, Schacter et al. examined the impact of the TAP System across 11 
schools. The same cluster level analysis with multi-level multivariate analyses were employed 
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using all available covariates to compare growth between the TAP and control schools. Results 
from the study indicate that 65% of the TAP schools outperformed their matched controls in 
reading, language, and mathematics achievement, with the magnitude of change ranging from 
6% to 46%. The teacher satisfaction component of this study indicated strong support for the 
four core principles of the TAP System. 


Following a consistent pattern of multiple researchers investigating TAP in different locations using 
varied methodological approaches, three new reports are highlighted and summarized below 
demonstrating TAP’s consistently positive impact on schools, teachers, and student achievement. 


New TAP Studies 
Evaluation of TAP System across Louisiana 


A study released in fall 2014 (Barnett, Wills, & Kirby, 2014) evaluated the impact of the TAP System 
across 66 schools in Louisiana. The study, Comprehensive Educator Effectiveness Models that Work: 
Impact of TAP: The System on Student Achievement in Louisiana, was conducted by researchers within 
NIET and a third-party external evaluator, Dr. Peggy Kirby, president of ed-cet, inc., a firm specializing in 
education program evaluation. The study of schools from across Louisiana included elementary, middle, 
and high schools in urban, suburban, and rural communities. The study examined the impact of the TAP 
System on student achievement using two rigorous analytic strategies. 


First, the authors employed a linear regression to compare 2012-13 Assessment Index (Al) performance 
of the 66 TAP schools and non-TAP schools statewide, controlling for prior (2010-11) achievement, 
percentage of students receiving free or reduced-price lunch, school configuration, school size (number 
of students), and percentage of English language learners. 


Second, the authors compared the TAP schools with a propensity score matched group of non-TAP 
schools. The authors matched on baseline (2010-11) student achievement, school configuration (grades 
taught), percentage of students receiving free or reduced-price lunch, and school size (number of 
students). The matched groups were compared on 2012-13 K-8 Al as the dependent variable. 


To answer the first question (Do students in TAP schools outperform students in similar schools 
statewide?) student performance was examined alongside a statewide group of comparison schools. 
Regression coefficients for all covariates were significant at p < .05 (n = 1003 schools with all covariate 
information), as shown in Table 1. Controlling for the covariates, implementation of the TAP System 
showed a significant positive effect on 2012-13 achievement: the 66 TAP schools scored 3.7 points 
higher on average than non-TAP schools (p < .01). 


Table 1: Coefficient Estimates for Regression with 2012-13 Louisiana Assessment Index 


Estimate | Standard error t p 
Intercept 19.252 3.558 5.41 .000*** 
2010-11 Al 0.786 0.022 36.16 .000*** 
% FRL -0.183 0.022 -8.27 .000*** 
School configuration 
(baseline is elementary): 
Middle/junior high school -1.669 0.786 -2.12 .034* 
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High school -4.009 0.984 -4.07 .000*** 


School size -0.003 0.001 -2.37 .018* 
% LEP 0.334 0.076 4.41 .000*** 
TAP school 3.699 1.221 3.03 .003** 


* p< .05. ** p< .01. *** p< .001. 


For the second research question, the authors selected one propensity score matched control for each 
of the 66 TAP schools. The covariates in this model were 2010-11 Al, school configuration, percentage of 
students receiving free or reduced-price lunch, and school size (number of students). To verify the 
quality of matching, the authors assessed the standardized differences in means of the covariates 
between the TAP schools and the matches; all differences were well below the maximum acceptable 
value of 0.25 (Stuart, 2010). Like the TAP schools, the matched group consisted of 47 elementary 
schools, 15 middle schools, and 4 combination schools that serve students in the K-8 range. 


The average 2012-13 Al for TAP schools (64.45) was 5.47 points greater than the average for the 
matched controls (58.98). To illustrate the gain in score for the TAP schools, Figure 6 shows the 
equivalent starting point for both TAP and control schools as a function of the maximum score on the 
2010-11 Al (maximum of 200) and the change in score relative to the maximum score on the 2012-13 Al 
(maximum of 150). The score comparison indicates the TAP schools made larger gains in student 
achievement than matched comparison schools. 


44% 


43% 43% 
42% 
41% 
Percentage of 40% 
maximum K-8 39% 39% 
Al 38% 
37% 37% 
36% 37% 
35% 
34% 
2010-11 2012-13 


e===—TAP schools e==<<=Matched schools 


Figure 6: 2012-13 K-8 Assessment Index averages 


Read the full Louisiana Impact report at http://www.niet.org/assets/Publications/louisiana-tap-student- 
achievement.pdf. 


TAP’s Effect on Teacher Retention 


Counter to the national trends that show greater teacher attrition levels in high-need schools, a study 
from NIET finds that campuses implementing TAP: The System for Teacher and Student Advancement 
are keeping effective teachers at a higher rate than similar high-need schools and the national average 
overall (Barnett & Hudgens, 2014). 
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The TAP System retains teachers by creating teacher leaders; instituting high-quality and relevant 
professional development; building a structure for educator observation that supports faculty members 
through a trajectory of continual improvement; and offering opportunities for pay based on educators’ 
roles and responsibilities, their accomplishments in the classroom, and the learning growth of their 
students. Specifically, the study—Staying Power: The Impact of the TAP System on Retaining Teachers 
Nationwide—reports that these methods have helped TAP schools retain approximately 14% more 
teachers than similar high-need schools and 10% more than the national average (Figure 7). 
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80% 
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75% 
70% 
National Average High-needs Schools TAP Schools 


Figure 7: Retention rates in TAP schools relative to national average and high-need schools. 


The study also examines the characteristics of different groups of teachers in TAP schools. The results of 
these analyses demonstrate that TAP teachers continue to improve while working in TAP schools. 
Specifically, the teachers in the TAP schools demonstrate growth in their instructional ability (TAP 
schools refer to this as Skills, Knowledge and Responsibilities or SKR scores). For stayers, SKR scores 
increased from 2010-11 (M = 3.32; SD = 0.54) to 2011-12 (M = 3.44; SD = 0.51), t(2761) = 14.42, p =.00, d 
= 0.23. Additionally, SKR scores increased from 2011-12 (M = 3.25; SD = 0.50) to 2012-13 (M = 3.39; SD = 
0.51), t(6412) = 28.84, p = .00, d=0.28. 


More importantly, this growth in instructional ability also occurred while student performance scores 
demonstrated growth, as measured by the gain in the classroom value-added scores from 2011-12 (M = 
3.19, SD = 1.16) to 2012-13 (M = 3.25; SD = 1.13), t(1604) = 2.03, p= .04, d= 0.05. 


To put these findings into perspective, across the eight comparisons—two groups, two outcome 
measures, two change years—TAP schools resulted in higher mean scores in seven of the eight 
comparisons. In all four comparisons of teacher instructional ability, the TAP teachers demonstrated 
significant increases (p < 0.05). Further, in three of the four classroom value-added comparisons, the 
TAP schools also showed increases. This result indicates the TAP teachers remaining in TAP schools were 
more effective each year. 


To also understand the degree of impact in retaining teachers, beyond a 14% comparative advantage, 
the effect size calculations provide further insight. The average effect size across the comparisons is d = 
0.26. This finding shows that the effect of the TAP System on retaining teachers is approximately equal 
to the effect size of reducing class sizes by one-third on student achievement (Coe, 2002). While these 
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are different outcome measures, they provide some relativity to showcase how substantial the effect of 
the TAP System is on keeping teachers in schools. 


To further explore the net effect of retaining teachers, and as demonstrated, more effective teachers, 
we examined the ongoing conversation about the costs of educator attrition for schools, teachers, 
students, and communities. Given the discrepancy in costs (ranging from $5,000 to 200% of the 
departing teacher’s salary), a conservative estimate of teacher attrition may be defined at $8,500 per 
teacher (see ACORN, 2004; Barnes, Crowe, & Schaefer, 2007; Simon & Johnson, 2013; Texas Center for 
Educational Research, 2000). 


To illustrate this impact, consider a representative elementary school with 40 teachers and a 20% 
attrition rate. In this school, eight teachers would depart the typical school each year. In a TAP System 
representative elementary school with 40 teachers, two teachers (6% attrition) would depart each year. 
The cost of attrition difference between these two schools on a conservative estimate would be 
$50,000?. Using higher estimates of attrition and larger schools, one can quickly glean the impact of the 
TAP System retention rate on the overall budget of a school when examining solely the attrition aspect. 
Non-pecuniary benefits (i.e. collegiality, morale, teacher satisfaction) would be expected to increase as a 
result of lower attrition as well. 


Read the full retention report at http://www.niet.org/assets/Publications/staying-power-tap-system- 
teacher-retention.pdf. 


TAP High School Summit 


Educators were drawn from TAP partnering high schools across the nation, including administrators and 
teachers from traditional public and charter schools, representing education agencies at the school, 
district, and state levels. The purpose of this meeting was to collect information from educators with 
regard to implementing the TAP System at the high school level, specifically focusing on “what makes 
the high school implementation different/unique.” Dozens of insights and solutions were discussed. The 
participants organized their insights around three key areas—cluster, field testing, and follow-up—and 
shared their experience in how the TAP System effectively impacts their high schools (see Figure 8 for 
list of topics). 


3 This example uses a conservative estimate in a representative non-TAP and TAP school, each comprised of 40 
teachers. In the non-TAP school, 8 teachers would leave each year (20% loss). In the TAP school, 2 teachers would 
leave each year (6% loss). The conservative estimate for each teacher lost is $8,500. Therefore, the net loss of 6 
teachers at the non-TAP school results in a cost of $50,000. 


National Institute for Excellence in Teaching Page 13 


Cluster at the Field Testing at the 
High School Level High School Level 


Follow-Up at the 
High School Level 


Shift from Content-Only Focus to 
Pedagogy 
Transfer from Cluster to Differentiation across Content 
Classroom and Grade Level 
Determining Which Student Using Student Work to Drive 
Work to Analyze Field Testing 


Figure 8: Key topics discussed at the High School Summit. 


Read a summary of the discussion and solutions from the High School Summit report at 
http://www.niet.org/assets/Publications/tap-high-school-symposium.pdf 


National TAP Attitude Surveys - 2014 
Teacher Results - 2014 TAP Attitude Survey 


Critics of performance incentives for teachers claim that they will result in competitiveness and a loss of 
collegiality among teachers. Notwithstanding, we find evidence of a high degree of collegiality in TAP 
schools. In the 2014 TAP national survey of teacher attitudes, 93% of teachers in TAP schools agree with 
statements reporting a high level of collegiality in their schools, and approximately 70% report strong 
agreement. This evidence for collegiality has been remarkably high over the last decade, as shown in 
Figure 9*, which indicates that TAP schools consistently have a collaborative and collegial environment. 


Teacher Collegiality 


Percentage 
@ Moderate Support G@ Strong Support 


(n=9,984 teachers, 2014 NIET Teacher Survey) 


Figure 9: Level of reported collegiality from teachers in TAP schools nationwide. 


* The five dimensions represented in this and the next figures are constructed from multiple teacher survey items 
using factor analysis. Most items in the survey are based on a 5-point Likert scale indicating agreement (1=Not at 
All and 5=Very Much). For reporting purposes on collegiality and the four TAP elements, the results are presented 
as Moderate (weighted average of 3 on the items for that factor) and Strong (weighted average of 4 or 5 on the 
items for that factor). 
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Beyond the overall high levels of reported collegiality within TAP schools, the levels of support for the 
specific elements of TAP including multiple career paths, ongoing applied professional growth, 
instructionally focused accountability and performance-based compensation are also high and growing, 
as shown in Figures 10-13. 


A growing number of teachers report that the creation of teacher leader roles in their school has a 
positive impact on student achievement and school goals. Teacher leaders and administrators form a 
leadership team that articulates school goals and supports each teacher in developing and achieving 
their own instructional goals based on their skills and their students’ needs. Teacher leadership roles 
also provide a pathway for teachers to make a greater contribution to the instructional excellence of a 
school without leaving the classroom. 


Multiple Career Paths 


0 25 50 75 100 
Percentage 


@ Moderate Support WStrong Support 


(n=9,984 teachers, 2014 NIET Teacher Survey) 


Figure 10: Level of reported support for multiple career paths from teachers nationwide. 


In TAP schools, master and mentor teachers lead weekly cluster group meetings where they examine 
student data, engage in collaborative planning, and discuss instructional strategies that have been field- 
tested in their own schools. Teachers benefit from access to a national TAP database of instructional 
strategies and their colleagues’ experiences. Professional development continues in the classroom as 
master teachers model lessons, observe classroom instruction and support teachers’ pedagogical 
improvement. Figure 11 demonstrates the increasing strong level of support for the professional growth 
that occurs in TAP schools, and further shows the sustained high level of overall support. 


Ongoing Applied Professional Growth 
a ae ean 
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(n=9,984 teachers, 2014 NIET Teacher Survey) 


Figure 11: Level of reported support for applied professional growth from teachers 
nationwide. 


TAP teachers are observed in classroom instruction multiple times a year by multiple trained observers, 
including principals and master and mentor teachers, using rubrics measuring indicators of instructional 
effectiveness. Evaluators are trained and certified, and leadership teams monitor the reliability and 
consistency of evaluations in their schools. These classroom evaluations are complemented by analyzing 
student achievement growth, rounding out a multi-measure system of teacher evaluation. Evaluation 
results are used as formative feedback in one-on-one mentoring sessions, and guide planning for cluster 
group meetings. Figure 12 illustrates the strong level of support for the instructionally focused 
accountability reported by TAP teachers across the nation. 


Instructionally Focused Accountability 


0 25 50 75 100 


Percentage 
Moderate Support WStrong Support 


(n=9,984 teachers, 2014 NIET Teacher Survey) 


Figure 12: Level of reported support for instructionally focused accountability from teachers 
nationwide. 


TAP teachers have the opportunity to earn annual bonuses based on their observed skills, knowledge 
and responsibilities, their students’ average achievement growth, and school-wide achievement growth. 
Master and mentor teachers receive additional compensation based on their added roles and 
responsibilities, and principals can earn additional compensation based on school-wide achievement 
growth and other measures of effectiveness. Figure 13 demonstrates the level of support for the 
performance-based compensation system from TAP teachers. Levels of support for performance-based 
compensation in TAP schools has increased significantly over time, from 49% in 2005 to 76% in 2014. 


National Institute for Excellence in Teaching Page 16 


Performance-Based Compensation 


0 25 50 75 100 
Percentage 


@ Moderate Support @ Strong Support 


(n=9,984 teachers, 2014 NIET Teacher Survey) 


Figure 13: Level of reported support for performance-based compensation from teachers 
nationwide. 
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Principal Results - 2014 TAP Attitude Survey 


The above substantially positive results from the TAP teacher survey are echoed by the 2014 TAP 
Principal Survey. As shown in Figure 14, principals have overwhelmingly reported that TAP has a positive 
effect on collegiality, instructional practices, and teacher effectiveness. Virtually all principals reported 
TAP’s professional growth activities increased teacher instructional practices. Further, as noted on 
Figure 15, a significant majority of principals (86%) agree that the TAP System helps retain effective 
teachers. 


The TAP professional growth activities improve my 
school's teachers’ instructional practice. 


The TAP professional growth activities lead to 
collegiality among my school's teachers. 


The TAP teacher evaluation system improves my 
school's teachers’ instructional practice. 


My teachers are more effective. 


0% 25% 50% 75% 100% 
Percentage of Administrators 


m Moderate m Strong 


(n=430 administrators, 2014 NIET Administrator Survey) 


Figure 14: Principal survey results on TAP outcomes in their schools. 


No, 5% 


Somewhat, 9% 


Yes, 86% 


(n=430 administrators, 2014 NIET Administrator Survey) 


Figure 15: “The implementation of TAP has helped retain effective teachers at my school.” 


National Institute for Excellence in Teaching Page 18 


Upcoming Projects 
Teacher and Administrator Attitude Survey 


The annual TAP Teacher and Administrator Attitude Survey measures the impact of TAP on teacher 
attitudes and job satisfaction. The survey contains questions that assess the perceptions of the 
implementation of TAP at the school and teachers’ and administrators’ overall job satisfaction. NIET 
asked approximately 14,000 teachers and 1,000 administrators within TAP schools across 11 states (i.e. 
AR, AZ, CA, CO, IA, IN, LA, MN, SC,TN, and TX) to complete the survey. The preceding results provide a 
glimpse into those responses aggregated across all respondents and comparing an early adoption year 
(2005) to the current year (2015). The next step in this analysis is to examine the changes across time 
and within each location, as well as examine additional questions reported throughout the survey, 
including the qualitative data and open-ended responses. 


Measurement Stability of TAP Rubric and Alignment with Value Added 


We are investigating multiple approaches to the stability of TAP observational rubric scores. One metric 
is the correlation between scores obtained from the same subjects at different time points. The 
correlations between average SKR scores in the 2010-11, 2011-12, and 2012-13 school years were highly 
significant and large, approaching 0.7 for consecutive school years. Another metric we investigated was 
the distribution of maximum year-to-year score changes. For more than 90% of teachers, scores either 
did not change over the period, or changed by 0.5. Score changes were within 1.0 for nearly 99% of 
teachers. Overall, the preliminary evidence—high year-to-year correlations, generally small year-to-year 
score changes, and a large proportion of score patterns consistent with improvement—is consistent 
with an instrument that generates stable scores while providing teachers with actionable feedback that 
guides them in improving their practice. 


In addition, we are examining the relationship between TAP rubric scores and the classroom value- 
added component of teacher evaluation. The correlation of TAP SKR scores with classroom value-added 
exceeds all correlations between observational and classroom value-added measures reported in the 
literature to date (with the exception of one reported correlation with an equal value). This is 
encouraging evidence that the TAP observation rubric measures aspects of teacher practice that 
contribute to improved student test performance. 


Career Ladders in Knox County, TN, TAP Schools 


NIET received a federal Teacher Incentive Fund grant (TIF3) in 2010 to expand TAP System 
implementation in Knox County, TN, schools. With assistance from NIET, a third-party external evaluator 
is examining the career trajectories of former TAP mentor and master teachers who have transitioned 
into administrative leadership positions. Through the use of interview, survey, and focus group data, the 
authors explore structures within TAP that promote teacher leadership and prepare administrators to 
be effective instructional leaders. 
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Conclusion 


The TAP System stands out because of its more than a decade-long track record of growth and success 
in raising student achievement in high-need schools. The research evidence also reveals several key 
reasons for TAP’s positive impact: an evaluation system capable of differentiating teacher performance 
levels and providing detailed feedback for improvement, ongoing professional growth in classroom 
practice using student and teacher data to guide improvement, recruitment and retention of effective 
teachers, and the creation of a challenging, rewarding, and collegial environment focused on high- 
quality instruction and student learning. 
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